Assisting Machine Learning Through Shaping, Advice and Examples
نویسندگان
چکیده
Many different methods for combining human expertise with machine learning in general, and evolutionary computation in particular, are possible. Which of these methods work best, and do they outperform human design and machine design alone? In order to answer this question, a human-subject experiment for comparing human-assisted machine learning methods was conducted. Three different approaches, i.e. advice, shaping, and demonstration, were employed to assist a powerful machine learning technique (neuroevolution) on a collection of agent training tasks, and contrasted with both a completely manual approach (scripting) and a completely hands-off one (neuroevolution alone). The results show that, (1) human-assisted evolution outperforms a manual scripting approach, (2) unassisted evolution performs consistently well across domains, and (3) different methods of assisting neuroevolution outperform unassisted evolution on different tasks. If done right, human-assisted neuroevolution can therefore be a powerful technique for constructing intelligent agents.
منابع مشابه
Towards an Ensemble Framework for Assisting in Synthesis Tasks
It is a common practice when problem solving to seek the advice of others as to what decision to make next. One may seek advice from a single trusted colleague, but commonly one seeks advice frommultiple sources, weighing the advice from each before making a decision (Polikar 2006). This behavior is mimicked by ensemble methods for machine learning which combine outputs from multiple independen...
متن کاملEvolving Robot Vision: Increasing Performance through Shaping
Automated methods for designing robot controllers based on machine-learning techniques have shown great promise when applied to simple robot tasks, but in order to ‘scale up’ to more complicated problems they will require assistance from human experts, a process that is often called ‘robot shaping’. In this paper, the difficult problem of learning how to visually track moving objects is examine...
متن کاملActive Advice Seeking for Inverse Reinforcement Learning
Intelligent systems that interact with humans typically require input in the form of demonstrations and/or advice for optimal decision making. In more traditional systems, such interactions require detailed and tedious effort on the part of the human expert. Alternatively, active learning systems allow for incremental acquisition of the demonstrations from the human expert where the learning sy...
متن کاملLearning to Make Good Use of Operational Advice
We address the problem of advice-taking in a given domain, in particular for building a gameplaying program. Our approach to solving it strives for the application of machine learning techniques throughout, i.e., for avoiding knowledge elicitation by any other means as much as possible. In particular, we build upon existing work on the operationalization of advice by machine and assume that adv...
متن کاملAdvice Taking and Transfer Learning: Naturally Inspired Extensions to Reinforcement Learning
Reinforcement learning (RL) is a machine learning technique with strong links to natural learning. However, it shares several “unnatural” limitations with many other successful machine learning algorithms. RL agents are not typically able to take advice or to adjust to new situations beyond the specific problem they are asked to learn. Due to limitations like these, RL remains slower and less a...
متن کامل